Supporting Individuals with Special Needs through Intelligent Visual Schedules

Interventions to support individuals with cognitive disabilities surrounding the organization of time and activities often include what are known as visual schedules. These artifacts use words, images, and sometimes tangible objects to represent activities that will take place (or have taken place) arranged in temporal order to augment understanding of time, events, and places. Use of visual schedules has been shown to reduce the symptoms associated with these disabilities, in particular for individuals with autism spectrum disorder (ASD). By further expanding these capabilities using interactive and intelligent systems, computing based visual scheduling systems can more fully realize the vision of alleviating the anxiety and disorder associated with unplanned and changing events. In this paper, we describe the motivation, approach, and plan for designing, developing, deploying, and evaluation such systems. Introduction and Motivation The structure needed to reduce anxiety and support better self-organization around time and activities for individuals with autism spectrum disorder (ASD) and other special needs is often provided through visual schedules. “Visual schedules display planned activities in symbols (words, pictures, photograph, icons, actual objects) that are understood in the order in which they will occur” (ICAN 2007). They present the abstract concepts of activities and time in concrete forms by using pictures, words, and other visual elements to describe what will happen, in what order, and where. They have been used successfully in classrooms, homes, and private practices to address difficulties with sequential memory, organization of time, and language comprehension and to lessen anxiety (Hodgdon, 1999; Mesibov et al., 2002; Savner & Myles, 2000). In schools, visual schedules can assist students with transitioning independently between activities and environments by telling them where to go and helping them to know what they will do when they get there. Figure 1: Visual schedules can be placed on the wall for group activities (left) or created for individual use in a mobile or carryable format (above). The personal “wallet” shown here was developed as part of an integrated communication strategy, wherein the teacher would use verbal cues coupled with visual cues to remind the student of his tasks and ask him which reward he would choose. Many experts in the design and use of paper-based visual schedules advocate a caregivers ca mple schedule...at n a piece of pa have a grid system...one box say s Then the TeachTown website). Visual schedu s broken down any experts, such n this... “ str e understanding of time and activities by s ose a larger activity and demonstrating visually a event at the end of the task. For example, “Handwashing” can be represented by “First turn on the water, then place your hands under the water”, etc. A caregiver might end the “handwashing” sequence with a picture of dinner, indicating that once “handwashing” is completed, the enjoyable activity of eating dinner will take place. The First/Then structure can also be used at a more macro level as in “First work, then play.” By providing structure, visual schedules reduce anxiety and support behavior intervention plans f severe behavior problems. They can also support individuals with less severe disabilities in entering the workplace by providing external direction for common workplace phenomena. Because the information must be kept up to date – an extremely onerous task – and the schedules themselves tend to be more effective when they are engaging to the individuals using them, the traditional pen and paper “low tech” assistive technology approach c edules can be even more useful and successful with the addition of omputing technologies. ping needs of Figure 2: This portable visual schedule demonstrates the "First/Then" model supported by many theories of designing visual schedules. First/Then approach. For example, Chris Whalen, Founder and Chief Science Officer at TeachTown, describes how n begin using simple schedules: “Provide a very si first. Make a grid with two boxes (this can be done o per, draw a line down the middle and then you s First this... and the other say this...” (Whalen 2006) (See also Figure 2, also from les can be used at the micro level supporting task into sub-elements. For this type of activity, m as those at TechTown advocate a “First this... The ucture. This structure serves to augment th howing both the sequence of events that comp reward or enjoyable ocused on students with an be improved. These sch interactive and intelligent c Background of the Researchers The two primary researchers on this work, Gillian R. Hayes and Donald J. Patterson, each individually focused on technologies related to augmented cognition as part of their dissertation work. Hayes completed her doctoral studies at the Georgia Institute of Technology. While there, she was supported by Cure Autism Now, the Organization for Autism Research, and through an BM PhD fellowship to focus her research on supporting the everyday record kee I caregivers for children with autism spectrum disorder. Her primary advisor was Gregory D. Abowd with whom she worked on CareLog and concepts of Selective Archiving. Patterson completed his doctoral studies at the University of Washington. While there, he was an NDSEG fellow and did research on artificial intelligence and its application to Alzheimer's Disease management and DNA biotechnology. His primary advisors were Henry Kautz and Dieter Fox with whom he worked on the Activity Compass also known as "Opportunity Knocks." Both are now Assistant Professors in the Department of Informatics at the University of California, Irvine focusing on ubiquitous computing and human computer interaction. Additionally, Patterson’s work continues to cross into artificial intelligence and machine learning while Hayes’s work ontinues to involve elements of collaborative and cooperative computing. c Our work includes projects focused on assisting caregivers of children with autism (Hayes & Abowd 2006, Kientz et al. 2007, Hayes et al. 2004), supporting the needs of elderly individuals both in their homes and while mobile (Liao et al.2004, Lundell & Hayes 2005, Patterson et al. 2006, Patterson et al. 2004, Patterson et al. 2002,), identifying paths and supporting wayfinding for a variety of individuals (Patterson et al. 2004) and supporting chronic care management (Abowd et al. 2006, Hayes et al. in preparation). These projects have been completed in cooperation with researchers at three Intel research sites and IBM TJ Watson as well as with the support of nonprofit and governmental organizations such as Cure Autism Now, the NSF, and the IH. Together, we are continuing threads of work in the areas of interactive, collaborative, and ividuals with special needs or in chronic care situations.